<!DOCTYPE html>
<html>
<head><script src="//archive.org/includes/analytics.js?v=cf34f82" type="text/javascript"></script>
<script type="text/javascript">window.addEventListener('DOMContentLoaded',function(){var v=archive_analytics.values;v.service='wb';v.server_name='wwwb-app227.us.archive.org';v.server_ms=216;archive_analytics.send_pageview({});});</script>
<script type="text/javascript" src="/_static/js/bundle-playback.js?v=1WaXNDFE" charset="utf-8"></script>
<script type="text/javascript" src="/_static/js/wombat.js?v=txqj7nKC" charset="utf-8"></script>
<script>window.RufflePlayer=window.RufflePlayer||{};window.RufflePlayer.config={"autoplay":"on","unmuteOverlay":"hidden"};</script>
<script type="text/javascript" src="/_static/js/ruffle.js" defer="true"></script>
<script type="text/javascript">
__wm.init("https://web.archive.org/web");
__wm.wombat("http://symas.com/mdb/microbench/","20160304015634","https://web.archive.org/","web","/_static/",
"1457056594");
</script>
<link rel="stylesheet" type="text/css" href="/_static/css/banner-styles.css?v=S1zqJCYt" />
<link rel="stylesheet" type="text/css" href="/_static/css/iconochive.css?v=qtvMKcIJ" />
<!-- End Wayback Rewrite JS Include -->
<title>Database Microbenchmarks</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link rel="shortcut icon" href="/web/20160304015634im_/http://symas.com/favicon.ico">
<style>
body {
font-family:Helvetica,sans-serif;
padding:20px;
}
h2 {
padding-top:30px;
}
table.bn {
width:800px;
border-collapse:collapse;
border:0;
padding:0;
}
table.bnbase {
width:650px;
}
table.bn td {
padding:2px 0;
}
table.bn td.c1 {
font-weight:bold;
width:150px;
}
table.bn td.c1 div.e {
float:right;
font-weight:normal;
}
table.bn td.c2 {
width:160px;
text-align:right;
padding:2px;
}
table.bn td.c3 {
width:350px;
}
table.bn td.c4 {
width:150px;
font-size:small;
padding-left:4px;
}
table.bn td.c5 {
text-align:right;
}
/* chart bars */
div.bldb {
background-color:#0255df;
}
div.bkct {
background-color:#df5555;
}
div.bsql {
background-color:#aadf55;
}
div.bmdb {
background-color:#ffa500;
}
div.bbdb {
background-color:#f00080;
}
.code {
font-family:monospace;
font-size:large;
}
.todo {
color: red;
}
</style>
</head>
<body><!-- BEGIN WAYBACK TOOLBAR INSERT -->
<script>__wm.rw(0);</script>
<div id="wm-ipp-base" lang="en" style="display:none;direction:ltr;">
<div id="wm-ipp" style="position:fixed;left:0;top:0;right:0;">
<div id="donato" style="position:relative;width:100%;">
<div id="donato-base">
<iframe id="donato-if" src="https://archive.org/includes/donate.php?as_page=1&platform=wb&referer=https%3A//web.archive.org/web/20160304015634/http%3A//symas.com/mdb/microbench/"
scrolling="no" frameborder="0" style="width:100%; height:100%">
</iframe>
</div>
</div><div id="wm-ipp-inside">
<div id="wm-toolbar" style="position:relative;display:flex;flex-flow:row nowrap;justify-content:space-between;">
<div id="wm-logo" style="/*width:110px;*/padding-top:12px;">
<a href="/web/" title="Wayback Machine home page"><img src="/_static/images/toolbar/wayback-toolbar-logo-200.png" srcset="/_static/images/toolbar/wayback-toolbar-logo-100.png, /_static/images/toolbar/wayback-toolbar-logo-150.png 1.5x, /_static/images/toolbar/wayback-toolbar-logo-200.png 2x" alt="Wayback Machine" style="width:100px" border="0" /></a>
</div>
<div class="c" style="display:flex;flex-flow:column nowrap;justify-content:space-between;flex:1;">
<form class="u" style="display:flex;flex-direction:row;flex-wrap:nowrap;" target="_top" method="get" action="/web/submit" name="wmtb" id="wmtb"><input type="text" name="url" id="wmtbURL" value="http://symas.com/mdb/microbench/" onfocus="this.focus();this.select();" style="flex:1;"/><input type="hidden" name="type" value="replay" /><input type="hidden" name="date" value="20160304015634" /><input type="submit" value="Go" />
</form>
<div style="display:flex;flex-flow:row nowrap;align-items:flex-end;">
<div class="s" id="wm-nav-captures" style="flex:1;">
<a class="t" href="/web/20160304015634*/http://symas.com/mdb/microbench/" title="See a list of every capture for this URL">82 captures</a>
<div class="r" title="Timespan for captures of this URL">03 Dec 2012 - 06 Dec 2022</div>
</div>
<div class="k">
<a href="" id="wm-graph-anchor">
<div id="wm-ipp-sparkline" title="Explore captures for this URL" style="position: relative">
<canvas id="wm-sparkline-canvas" width="700" height="27" border="0"></canvas>
</div>
</a>
</div>
</div>
</div>
<div class="n">
<table>
<tbody>
<!-- NEXT/PREV MONTH NAV AND MONTH INDICATOR -->
<tr class="m">
<td class="b" nowrap="nowrap"><a href="https://web.archive.org/web/20160113105325/http://symas.com/mdb/microbench/" title="13 Jan 2016"><strong>Jan</strong></a></td>
<td class="c" id="displayMonthEl" title="You are here: 01:56:34 Mar 04, 2016">MAR</td>
<td class="f" nowrap="nowrap"><a href="https://web.archive.org/web/20160405015250/http://symas.com:80/mdb/microbench" title="05 Apr 2016"><strong>Apr</strong></a></td>
</tr>
<!-- NEXT/PREV CAPTURE NAV AND DAY OF MONTH INDICATOR -->
<tr class="d">
<td class="b" nowrap="nowrap"><a href="https://web.archive.org/web/20160113105325/http://symas.com/mdb/microbench/" title="10:53:25 Jan 13, 2016"><img src="/_static/images/toolbar/wm_tb_prv_on.png" alt="Previous capture" width="14" height="16" border="0" /></a></td>
<td class="c" id="displayDayEl" style="width:34px;font-size:22px;white-space:nowrap;" title="You are here: 01:56:34 Mar 04, 2016">04</td>
<td class="f" nowrap="nowrap"><a href="https://web.archive.org/web/20160305073246/http://symas.com:80/mdb/microbench" title="07:32:46 Mar 05, 2016"><img src="/_static/images/toolbar/wm_tb_nxt_on.png" alt="Next capture" width="14" height="16" border="0" /></a></td>
</tr>
<!-- NEXT/PREV YEAR NAV AND YEAR INDICATOR -->
<tr class="y">
<td class="b" nowrap="nowrap"><a href="https://web.archive.org/web/20150221015749/http://symas.com:80/mdb/microbench/" title="21 Feb 2015"><strong>2015</strong></a></td>
<td class="c" id="displayYearEl" title="You are here: 01:56:34 Mar 04, 2016">2016</td>
<td class="f" nowrap="nowrap"><a href="https://web.archive.org/web/20170707180249/http://symas.com/mdb/microbench/" title="07 Jul 2017"><strong>2017</strong></a></td>
</tr>
</tbody>
</table>
</div>
<div class="r" style="display:flex;flex-flow:column nowrap;align-items:flex-end;justify-content:space-between;">
<div id="wm-btns" style="text-align:right;height:23px;">
<span class="xxs">
<div id="wm-save-snapshot-success">success</div>
<div id="wm-save-snapshot-fail">fail</div>
<a id="wm-save-snapshot-open" href="#" title="Share via My Web Archive" >
<span class="iconochive-web"></span>
</a>
<a href="https://archive.org/account/login.php" title="Sign In" id="wm-sign-in">
<span class="iconochive-person"></span>
</a>
<span id="wm-save-snapshot-in-progress" class="iconochive-web"></span>
</span>
<a class="xxs" href="http://faq.web.archive.org/" title="Get some help using the Wayback Machine" style="top:-6px;"><span class="iconochive-question" style="color:rgb(87,186,244);font-size:160%;"></span></a>
<a id="wm-tb-close" href="#close" style="top:-2px;" title="Close the toolbar"><span class="iconochive-remove-circle" style="color:#888888;font-size:240%;"></span></a>
</div>
<div id="wm-share" class="xxs">
<a href="/web/20160304015634/http://web.archive.org/screenshot/http://symas.com/mdb/microbench/"
id="wm-screenshot"
title="screenshot">
<span class="wm-icon-screen-shot"></span>
</a>
<a href="#" id="wm-video" title="video">
<span class="iconochive-movies"></span>
</a>
<a id="wm-share-facebook" href="#" data-url="https://web.archive.org/web/20160304015634/http://symas.com/mdb/microbench/" title="Share on Facebook" style="margin-right:5px;" target="_blank"><span class="iconochive-facebook" style="color:#3b5998;font-size:160%;"></span></a>
<a id="wm-share-twitter" href="#" data-url="https://web.archive.org/web/20160304015634/http://symas.com/mdb/microbench/" title="Share on Twitter" style="margin-right:5px;" target="_blank"><span class="iconochive-twitter" style="color:#1dcaff;font-size:160%;"></span></a>
</div>
<div style="padding-right:2px;text-align:right;white-space:nowrap;">
<a id="wm-expand" class="wm-btn wm-closed" href="#expand" onclick="__wm.ex(event);return false;"><span id="wm-expand-icon" class="iconochive-down-solid"></span> <span class="xxs" style="font-size:80%;">About this capture</span></a>
</div>
</div>
</div>
<div id="wm-capinfo" style="border-top:1px solid #777;display:none; overflow: hidden">
<div id="wm-capinfo-notice" source="api"></div>
<div id="wm-capinfo-collected-by">
<div style="background-color:#666;color:#fff;font-weight:bold;text-align:center">COLLECTED BY</div>
<div style="padding:3px;position:relative" id="wm-collected-by-content">
<div style="display:inline-block;vertical-align:top;width:50%;">
<span class="c-logo" style="background-image:url(https://archive.org/services/img/webwidecrawl);"></span>
Organization: <a style="color:#33f;" href="https://archive.org/details/webwidecrawl" target="_new"><span class="wm-title">Internet Archive</span></a>
<div style="max-height:75px;overflow:hidden;position:relative;">
<div style="position:absolute;top:0;left:0;width:100%;height:75px;background:linear-gradient(to bottom,rgba(255,255,255,0) 0%,rgba(255,255,255,0) 90%,rgba(255,255,255,255) 100%);"></div>
The Internet Archive discovers and captures web pages through many different web crawls.
At any given time several distinct crawls are running, some for months, and some every day or longer.
View the web archive through the <a href="http://archive.org/web/web.php">Wayback Machine</a>.
</div>
</div>
<div style="display:inline-block;vertical-align:top;width:49%;">
<span class="c-logo" style="background-image:url(https://archive.org/services/img/hackernews00000)"></span>
<div>Collection: <a style="color:#33f;" href="https://archive.org/details/hackernews00000" target="_new"><span class="wm-title">Hackernews crawl number 0</span></a></div>
<div style="max-height:75px;overflow:hidden;position:relative;">
<div style="position:absolute;top:0;left:0;width:100%;height:75px;background:linear-gradient(to bottom,rgba(255,255,255,0) 0%,rgba(255,255,255,0) 90%,rgba(255,255,255,255) 100%);"></div>
Hacker News Crawl of their links.
</div>
</div>
</div>
</div>
<div id="wm-capinfo-timestamps">
<div style="background-color:#666;color:#fff;font-weight:bold;text-align:center" title="Timestamps for the elements of this page">TIMESTAMPS</div>
<div>
<div id="wm-capresources" style="margin:0 5px 5px 5px;max-height:250px;overflow-y:scroll !important"></div>
<div id="wm-capresources-loading" style="text-align:left;margin:0 20px 5px 5px;display:none"><img src="/_static/images/loading.gif" alt="loading" /></div>
</div>
</div>
</div></div></div></div><div id="wm-ipp-print">The Wayback Machine - https://web.archive.org/web/20160304015634/http://symas.com/mdb/microbench/</div>
<script type="text/javascript">//<![CDATA[
__wm.bt(700,27,25,2,"web","http://symas.com/mdb/microbench/","20160304015634",1996,"/_static/",["/_static/css/banner-styles.css?v=S1zqJCYt","/_static/css/iconochive.css?v=qtvMKcIJ"], false);
__wm.rw(1);
//]]></script>
<!-- END WAYBACK TOOLBAR INSERT -->
<a href="/web/20160304015634/http://symas.com/"><img src="/web/20160304015634im_/http://symas.com/images/new-symas-logo.png"></a><br>
<h1>Database Microbenchmarks</h1>
<p><a href="https://web.archive.org/web/20160304015634/http://www.symas.com/">Symas Corp.</a>, September 2012</p>
<hr>
<p>This page follows on from Google's LevelDB benchmarks published in July 2011 at
<a href="https://web.archive.org/web/20160304015634/http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html">LevelDB</a>. (A snapshot
of that document is available <a href="benchmark.html">here</a> for reference.
In addition to the benchmarks
tested there, we add the venerable BerkeleyDB as well as the OpenLDAP MDB database. For this test, we compare
LevelDB version 1.5 (git rev dd0d562b4d4fbd07db6a44f9e221f8d368fee8e4), <a href="https://web.archive.org/web/20160304015634/http://www.sqlite.org/">SQLite3</a>
(version 3.7.7.1) and <a href="https://web.archive.org/web/20160304015634/http://fallabs.com/kyotocabinet/spex.html">Kyoto Cabinet's</a> (version 1.2.76) TreeDB (a B+Tree based key-value store), <a href="https://web.archive.org/web/20160304015634/http://www.oracle.com/technetwork/products/berkeleydb/downloads/index.html">Berkeley DB 5.3.21</a>, and <a href="https://web.archive.org/web/20160304015634/http://www.openldap.org/">OpenLDAP MDB</a> (git rev 2e677bcb995b63d36461eea254f2134ebfe29da2). We would like to acknowledge the LevelDB project for the original benchmark code.</p>
<p>This page shows updated results for MDB write performance when using its
writable mmap option. The previous benchmark report is available <a href="july/">here</a>. Using the memory map
in read/write mode allows much faster writes and even lower memory overhead, but also opens
the possibility of undetected database corruption if bugs in the application cause overwrites
of arbitrary ranges of the map.</p>
<p>The writable mmap option was implemented after discussions at the LinuxCon in August. The results
here obsolete the results in the August LinuxCon presentation.</p>
<p>Benchmarks were all performed on a Dell Precision M4400 laptop with a quad-core Intel(R) Core(TM)2 ExtremeCPU Q9300 @ 2.53GHz, with 6144 KB of total L3 cache and 8 GB of DDR2 RAM at 800 MHz. (Note that LevelDB uses at most two CPUs since the benchmarks are single threaded: one to run the benchmark, and one for background compactions.)
The benchmarks were run on three different filesystems: tmpfs, reiserfs on an SSD, and ext2 on an HDD.
(See our <a href="july/">previous report</a> for performance tests on btrfs, ext2, ext3, ext4, jfs,
ntfs, reiserfs, xfs, and zfs.)
The SSD is a Crucial M4 512GB, purchased new in August 2012. The HDD is a Seagate ST9500420AS Momentus 7200.4
500GB notebook hard drive.
The system had Ubuntu 12.04 installed, with
kernel 3.5.0-030500. Tests were all run in single-user mode to prevent variations due to other system activity.
CPU performance scaling was disabled (scaling_governor = performance), to ensure a consistent CPU clock speed
for all tests. The numbers reported below are the median of three measurements. The databases are completely deleted
between each of the three measurements.</p>
<h4>Benchmark Source Code</h4>
<p>We wrote benchmark tools for SQLite, BerkeleyDB, MDB, and Kyoto TreeDB based on LevelDB's <span class="code">db_bench</span>.
The LevelDB, SQLite3, and TreeDB benchmark programs were originally provided in the LevelDB source distribution
but we've made additional fixes to the versions used here.
The code for each of the benchmarks resides here:</p>
<ul>
<li> <b>LevelDB:</b> <a href="db_bench.cc">db_bench.cc</a>.</li>
<li> <b>SQLite:</b> <a href="db_bench_sqlite3.cc">db_bench_sqlite3.cc</a>.</li>
<li> <b>Kyoto TreeDB:</b> <a href="db_bench_tree_db.cc">db_bench_tree_db.cc</a>.</li>
<li> <b>OpenLDAP MDB:</b> <a href="db_bench_mdb.cc">db_bench_mdb.cc</a>.</li>
<li> <b>BerkeleyDB:</b> <a href="db_bench_bdb.cc">db_bench_bdb.cc</a>.</li>
<li> <b>util/random.h:</b> <a href="random.h">random.h</a>.</li>
</ul>
<p>Google's original benchmark code had a number of flaws, as we noted in
<a href="https://web.archive.org/web/20160304015634/https://groups.google.com/d/msg/leveldb/9Ol2Gi4Yv6I/ZI5tH1ThfoUJ">this post
to the LevelDB Discussion Group</a>. We've fixed these flaws for these tests:
<ul>
<li>The Random write tests now use a shuffled list of all the keys, so no duplicate keys are used.
Thus the resulting databases should have the same content as from the Sequential tests.</li>
<li>The synchronous tests on tmpfs are identical in length to the asynch tests, to ensure
that the results are comparable. (We still shortened them on SSD and HDD due to their lengthy runtimes.)</li>
</ul>
</p>
<p>In addition, we found that the Kyoto TreeDB results were being unfairly penalized
relative to the LevelDB results, because all of its data items were undergoing an extra
alloc/copy before being sent to the database. The data items for LevelDB are now similarly
copied before use, to compensate for this issue. It's not clear how relevant this flaw
is in real world applications but it definitely skewed write speeds in LevelDB's favor
in these benchmarks.</p>
<h4>Custom Build Specifications</h4>
Compression support was disabled in the libraries that support it. No special malloc library was used in the build. All
of the benchmark programs were linked to their respective static libraries, to show the actual size needed for a
minimal program using each library.
<ul>
<li>LevelDB: Assertions were disabled.</li>
<li>TreeDB: We enabled the TSMALL and TLINEAR options when opening the database in order to reduce the footprint of each record.</li>
<li>SQLite: We tuned SQLite's performance, by setting its locking mode to exclusive. We also enabled SQLite's <a href="https://web.archive.org/web/20160304015634/http://www.sqlite.org/draft/wal.html">write-ahead logging</a>.</li>
<li>BerkeleyDB: We configure with --with-mutex=POSIX/pthreads to avoid using the default hybrid mutex implementation.</li>
<li>MDB: Assertions were disabled.</li>
</ul>
<a name="sec1">
<h2>1. Relative Footprint</h2></a>
<p>Most database vendors claim their product is fast and lightweight. Looking at the total size
of each application gives some insight into the footprint of each database implementation.
<pre>
size db_bench*
text data bss dec hex filename
272247 1456 328 274031 42e6f db_bench
1675911 2288 304 1678503 199ca7 db_bench_bdb
90423 1508 304 92235 1684b db_bench_mdb
653480 7768 1688 662936 a1d98 db_bench_sqlite3
296572 4808 1096 302476 49d8c db_bench_tree_db
</pre>
The core of the MDB code is barely 32K of x86-64 object code. It fits entirely within most modern CPUs' on-chip caches.
All of the other libraries are several times larger.
<a name="sec2">
<h2>2. Baseline Performance</h2></a>
<p>This section gives the baseline performance of all the
databases. Following sections show how performance changes as various
parameters are varied. For the baseline:</p>
<ul>
<li> All operations are running on tmpfs. This shows the pure CPU time each
database requires, independent of I/O speed.</li>
<li> Each database is allowed 4 MB of cache memory. (MDB has no cache, so this is irrelevant.)</li>
<li> Databases are opened in <em>asynchronous</em> write mode.
(LevelDB's sync option, TreeDB's OAUTOSYNC option,
SQLite3's synchronous options are all turned off, MDB uses the
MDB_NOSYNC option, and BerkeleyDB uses the DB_TXN_WRITE_NOSYNC option). I.e.,
every write is pushed to the operating system, but the
benchmark does not wait for the write to reach the disk.</li>
<li> Keys are 16 bytes each.</li>
<li> Values are 100 bytes each.</li>
<li> Sequential reads/writes traverse the key space in increasing order.</li>
<li> Random reads/writes traverse the key space in random order. The entire key space is shuffled so that every key is visited once; there are no duplicates or collisions in the random sequence.</li>
</ul>
<h3>A. Sequential Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">4,587,156 ops/sec</td>
<td class="c3"><div class="bldb" style="width:109px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">836,820 ops/sec</td>
<td class="c3"><div class="bkct" style="width:19px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">313,283 ops/sec</td>
<td class="c3"><div class="bsql" style="width:7px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">14,705,882 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">827,130 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:19px"> </div></td>
</table>
<h3>B. Random Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">174,246 ops/sec</td>
<td class="c3"><div class="bldb" style="width:81px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">103,252 ops/sec</td>
<td class="c3"><div class="bkct" style="width:48px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">82,994 ops/sec</td>
<td class="c3"><div class="bsql" style="width:38px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">751,315 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">101,958 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:47px"> </div></td>
</table>
<h3>C. Sequential Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">498,753 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">336,474 ops/sec</td>
<td class="c3"><div class="bkct" style="width:236px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">56,693 ops/sec</td>
<td class="c3"><div class="bsql" style="width:39px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">461,255 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:323px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">90,959 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:63px"> </div></td>
</table>
<h3>D. Random Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">317,662 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">96,984 ops/sec</td>
<td class="c3"><div class="bkct" style="width:106px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">42,199 ops/sec</td>
<td class="c3"><div class="bsql" style="width:46px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">240,154 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:264px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">45,928 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:50px"> </div></td>
</table>
<p>MDB has the fastest read operations by a huge margin, due to its single-level-store
architecture. MDB's write performance is good but not the fastest, due to the overhead of its
copy-on-write design. This copy overhead is most significant on small records like those
used in this test.</p>
<h3>E. Batch Writes</h3>
<p>A batch write is a set of writes that are applied atomically to the underlying database. A single batch of N writes may be significantly faster than N individual writes. The following benchmark writes one thousand batches where each batch contains one thousand 100-byte values. TreeDB does not support batch writes so its baseline numbers are
repeated here for reference.</p>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">677,048 entries/sec</td>
<td class="c3"><div class="bldb" style="width:95px"> </div></td>
<td class="c4">(1.36x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">336,474 entries/sec</td>
<td class="c3"><div class="bkct" style="width:47px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">109,302 entries/sec</td>
<td class="c3"><div class="bsql" style="width:15px"> </div></td>
<td class="c4">(1.93x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">2,481,390 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(5.38x baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">187,829 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:26px"> </div></td>
<td class="c4">(2.06x baseline)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">432,152 entries/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(1.36x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">96,984 entries/sec</td>
<td class="c3"><div class="bkct" style="width:79px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">58,432 entries/sec</td>
<td class="c3"><div class="bsql" style="width:37px"> </div></td>
<td class="c4">(1.38x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">294,898 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:115px"> </div></td>
<td class="c4">(1.23x baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">57,680 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:45px"> </div></td>
<td class="c4">(1.26x baseline)</td></tr>
</table>
<p>Batching allows some of MDB's copy-on-write overhead to be amortized across
multiple records. MDB's Append Mode for sequential writes is
most effective in batched operation.
</p>
<h3>F. Synchronous Writes</h3>
<p>In the following benchmark, we enable the synchronous writing modes
of all of the databases. Otherwise the test is identical to the Baseline.</p>
<ul>
<li>For LevelDB, we set WriteOptions.sync = true.</li>
<li>In TreeDB, we enabled TreeDB's OAUTOSYNC option.</li>
<li>For SQLite3, we set "PRAGMA synchronous = FULL".</li>
<li>For MDB, we set no options since full sync is its default mode.</li>
<li>For BerkeleyDB, we set no options since full sync is its default mode.</li>
</ul>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">444,247 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.89x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,730 ops/sec</td>
<td class="c3"><div class="bkct" style="width:4px"> </div></td>
<td class="c4">(0.017x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">51,308 ops/sec</td>
<td class="c3"><div class="bsql" style="width:40px"> </div></td>
<td class="c4">(0.91x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">404,694 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:318px"> </div></td>
<td class="c4">(0.88x baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">86,745 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:68px"> </div></td>
<td class="c4">(0.95x baseline)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">304,692 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.96x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5.698 ops/sec</td>
<td class="c3"><div class="bkct" style="width:6px"> </div></td>
<td class="c4">(0.058x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">38,713 ops/sec</td>
<td class="c3"><div class="bsql" style="width:44px"> </div></td>
<td class="c4">(0.92x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">224,871 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:258px"> </div></td>
<td class="c4">(0.94x baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">44,893 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:51px"> </div></td>
<td class="c4">(0.98x baseline)</td></tr>
</table>
<p>While most of the databases are only slowed down a little,
TreeDB performs extremely poorly in synchronous mode.</p>
<h3>G. Disk Usage</h3>
<p>The amount of disk space each database consumed is also a factor when
considering the footprint of each database. The results for each write test
are presented here, in KBytes used per test. The results are obtained using
du on the test directory at the completion of each test, after all data has
been flushed to disk. It includes space used by log files as well as actual
data files, if any.</p>
<table class="bn">
<tr><th></th><th colspan="2">Plain Writes</th><th colspan="2">Batched</th><th colspan="2">Synchronous</th></tr>
<tr><th></th>
<th>Sequential</th><th>Random</th>
<th>Sequential</th><th>Random</th>
<th>Sequential</th><th>Random</th></tr>
<tr><td class="c1">LevelDB</td>
<td class="c5">112692</td><td class="c5">114960</td><td class="c5">112348</td><td class="c5">115412</td><td class="c5">112692</td><td class="c5">114040</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c5">133104</td><td class="c5">229644</td><td class="c5"></td><td class="c5"></td><td class="c5">261980</td><td class="c5">300676</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c5">159128</td><td class="c5">158772</td><td class="c5">159232</td><td class="c5">160092</td><td class="c5">159136</td><td class="c5">158792</td></tr>
<tr><td class="c1">MDB</td>
<td class="c5">126660</td><td class="c5">180716</td><td class="c5">126660</td><td class="c5">192560</td><td class="c5">126660</td><td class="c5">180840</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c5">570400</td><td class="c5">683100</td><td class="c5">527436</td><td class="c5">639728</td><td class="c5">570400</td><td class="c5">682636</td></tr>
</table>
<p>LevelDB is the most efficient for disk usage, even without compression. We chose to
omit the built in compression from these tests because it is clearly not a core feature
of a database; any compression library could easily be used to handle compression for
any database using simple wrappers around their put and get APIs.</p>
<p>MDB's sequential write packing gives the most deterministic file sizes; the DB size
is the same regardless of batched or unbatched or synchronous writing. All of the other
databases and modes yield different sizes due to varying overheads in each mode.</p>
<p>BerkeleyDB's transaction log files form the bulk of its disk usage. We configured it
with DB_LOG_AUTO_REMOVE to run these tests, otherwise the consumption would be even higher.
In regular deployments it may be unsafe to use this setting, since removal of the log files
may prevent database recovery from working after a system crash. BerkeleyDB's transaction
log management has been a continual inconvenience in real world deployments.</p>
<a name="sec3">
<h2>3. Performance Using More Memory</h2></a>
<p>We increased the overall cache size for each database to 128 MB.
For SQLite3, we kept the page size at 1024 bytes, but increased the number of pages to 131,072 (up from 4096). For TreeDB, we also kept the page size at 1024 bytes, but increased the cache size to 128 MB (up from 4 MB).
For MDB there is no cache, so the numbers are simply a copy of the baseline. Both MDB and BerkeleyDB use
the default system page size (4096 bytes).</p>
<h3>A. Sequential Reads</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">4,566,210 ops/sec</td>
<td class="c3"><div class="bldb" style="width:108px"> </div></td>
<td class="c4">(0.99x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">1,242,236 ops/sec</td>
<td class="c3"><div class="bkct" style="width:29px"> </div></td>
<td class="c4">(1.48x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">441,501 ops/sec</td>
<td class="c3"><div class="bsql" style="width:10px"> </div></td>
<td class="c4">(1.41x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">14,705,882 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">860,585 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:20px"> </div></td>
<td class="c4">(1.04x baseline)</td></tr>
</table>
<h3>B. Random Reads</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">177,557 ops/sec</td>
<td class="c3"><div class="bldb" style="width:82px"> </div></td>
<td class="c4">(1.02x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">195,198 ops/sec</td>
<td class="c3"><div class="bkct" style="width:90px"> </div></td>
<td class="c4">(1.89x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">105,697 ops/sec</td>
<td class="c3"><div class="bsql" style="width:49px"> </div></td>
<td class="c4">(1.27x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">751,315 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">169,348 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:79px"> </div></td>
<td class="c4">(1.66x baseline)</td></tr>
</table>
<h3>C. Sequential Writes</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">488,520 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.98x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">323,939 ops/sec</td>
<td class="c3"><div class="bkct" style="width:232px"> </div></td>
<td class="c4">(0.96x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">54,660 ops/sec</td>
<td class="c3"><div class="bsql" style="width:39px"> </div></td>
<td class="c4">(0.96x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">461,255 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:330px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">91,525 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:65px"> </div></td>
<td class="c4">(1.006x baseline)</td></tr>
</table>
<h3>D. Random Writes</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">318,370 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(1.002x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">182,983 ops/sec</td>
<td class="c3"><div class="bkct" style="width:201px"> </div></td>
<td class="c4">(1.89x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">43,687 ops/sec</td>
<td class="c3"><div class="bsql" style="width:48px"> </div></td>
<td class="c4">(1.04x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">240,154 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:264px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">67,838 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:74px"> </div></td>
<td class="c4">(1.48x baseline)</td></tr>
</table>
<h3>E. Batch Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">683,995 entries/sec</td>
<td class="c3"><div class="bldb" style="width:96px"> </div></td>
<td class="c4">(1.40x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">324,675 entries/sec</td>
<td class="c3"><div class="bkct" style="width:45px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">105,119 entries/sec</td>
<td class="c3"><div class="bsql" style="width:14px"> </div></td>
<td class="c4">(1.92x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">2,481,390 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(5.38x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">191,534 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:27px"> </div></td>
<td class="c4">(2.09x non-batched)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">441,501 entries/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(1.39x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">182,983 entries/sec</td>
<td class="c3"><div class="bkct" style="width:145px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">64,078 entries/sec</td>
<td class="c3"><div class="bsql" style="width:50px"> </div></td>
<td class="c4">(1.47x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">294,898 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:233px"> </div></td>
<td class="c4">(1.23x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">104,613 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:82px"> </div></td>
<td class="c4">(1.54x non-batched)</td></tr>
</table>
<h3>F. Synchronous Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">441,696 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.89x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,730 ops/sec</td>
<td class="c3"><div class="bkct" style="width:4px"> </div></td>
<td class="c4">(0.017x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">51,243 ops/sec</td>
<td class="c3"><div class="bsql" style="width:40px"> </div></td>
<td class="c4">(0.90x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">404,694 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:320px"> </div></td>
<td class="c4">(0.88x baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">86,957 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:68px"> </div></td>
<td class="c4">(0.96x baseline)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">304,599 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.96x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,698 ops/sec</td>
<td class="c3"><div class="bkct" style="width:6px"> </div></td>
<td class="c4">(0.058x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">40,510 ops/sec</td>
<td class="c3"><div class="bsql" style="width:46px"> </div></td>
<td class="c4">(0.96x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">224,871 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:258px"> </div></td>
<td class="c4">(0.94x baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">65,789 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:75px"> </div></td>
<td class="c4">(1.43x baseline)</td></tr>
</table>
<p>Performance gains from the increased cache are minimal at best; some of the
tests are slower with the increased caches. Overall, application-level caching exacts
a great cost in terms of software and administrative complexity, and yields little
(if any) benefit in return.</p>
<a name="sec4">
<h2>4. Performance Using Large Values</h2></a>
<p>For this benchmark, we use 100,000 byte values. To keep the benchmark running time reasonable, we stop after writing 10,000 values. Otherwise, all of the same tests as for the Baseline are run.</p>
<h3>A. Sequential Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">299,133 ops/sec</td>
<td class="c3"><div class="bldb" style="width:3px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">16,514 ops/sec</td>
<td class="c3"><div class="bkct" style="width:0px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">7,402 ops/sec</td>
<td class="c3"><div class="bsql" style="width:0px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">30,303,030 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">9,133 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:0px"> </div></td>
</table>
<h3>B. Random Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">15,183 ops/sec</td>
<td class="c3"><div class="bldb" style="width:3px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">14,518 ops/sec</td>
<td class="c3"><div class="bkct" style="width:2px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">7,047 ops/sec</td>
<td class="c3"><div class="bsql" style="width:1px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">1,718,213 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">8,646 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:1px"> </div></td>
</table>
<p>
MDB's single-level-store architecture clearly outclasses all of the other designs; the others
barely even register on the results. MDB's zero-memcpy reads mean its read rate is
essentially independent of the size of the data items being fetched; it is only affected by the
total number of keys in the database.</p>
<h3>C. Sequential Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">3,366 ops/sec</td>
<td class="c3"><div class="bldb" style="width:91px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,860 ops/sec</td>
<td class="c3"><div class="bkct" style="width:158px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">2,029 ops/sec</td>
<td class="c3"><div class="bsql" style="width:55px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">12,905 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">1,920 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:52px"> </div></td>
</table>
<h3>D. Random Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">742 ops/sec</td>
<td class="c3"><div class="bldb" style="width:20px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,709 ops/sec</td>
<td class="c3"><div class="bkct" style="width:158px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">2,004 ops/sec</td>
<td class="c3"><div class="bsql" style="width:55px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">12,735 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">1,902 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:52px"> </div></td>
</table>
<h3>E. Batch Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">3,138 entries/sec</td>
<td class="c3"><div class="bldb" style="width:83px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,860 entries/sec</td>
<td class="c3"><div class="bkct" style="width:155px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">2,068 entries/sec</td>
<td class="c3"><div class="bsql" style="width:54px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">13,215 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">1,952 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:51px"> </div></td>
</table>
<h4>Random Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">3,079 entries/sec</td>
<td class="c3"><div class="bldb" style="width:82px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,709 entries/sec</td>
<td class="c3"><div class="bkct" style="width:152px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">2,041 entries/sec</td>
<td class="c3"><div class="bsql" style="width:54px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">13,099 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">1,939 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:51px"> </div></td>
</table>
<p>Unlike the other databases, MDB handles large value writes with ease.</p>
<p>In Google's original test only 1,000 entries are used, but that is too few
to characterize each database. With 10,000 entries, we see that LevelDB's
performance drops off a cliff in the random write tests. This is one of the
most common problems with microbenchmarks - very often the data sets are too
small to yield any meaningful results.</p>
<h3>F. Synchronous Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">3,368 ops/sec</td>
<td class="c3"><div class="bldb" style="width:91px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">3,121 ops/sec</td>
<td class="c3"><div class="bkct" style="width:84px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">2,026 ops/sec</td>
<td class="c3"><div class="bsql" style="width:54px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">12,916 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">1,913 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:51px"> </div></td>
</table>
<h4>Random Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">745 ops/sec</td>
<td class="c3"><div class="bldb" style="width:20px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">2,162 ops/sec</td>
<td class="c3"><div class="bkct" style="width:59px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">1,996 ops/sec</td>
<td class="c3"><div class="bsql" style="width:55px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">12,665 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">1,893 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:52px"> </div></td>
</table>
<p>None of the other databases even begin to approach MDB's performance when using
large data values.</p>
<h3>G. Disk Usage</h3>
<table class="bn">
<tr><th></th><th colspan="2">Plain Writes</th><th colspan="2">Batched</th><th colspan="2">Synchronous</th></tr>
<tr><th></th>
<th>Sequential</th><th>Random</th>
<th>Sequential</th><th>Random</th>
<th>Sequential</th><th>Random</th></tr>
<tr><td class="c1">LevelDB</td>
<td class="c5">979580</td><td class="c5">993188</td><td class="c5">977336</td><td class="c5">1109824</td><td class="c5">979580</td><td class="c5">990068</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c5">978076</td><td class="c5">978024</td><td class="c5"></td><td class="c5"></td><td class="c5">978232</td><td class="c5">978172</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c5">985812</td><td class="c5">985828</td><td class="c5">1082016</td><td class="c5">1083112</td><td class="c5">985832</td><td class="c5">985836</td></tr>
<tr><td class="c1">MDB</td>
<td class="c5">1000380</td><td class="c5">1000516</td><td class="c5">1000380</td><td class="c5">1001352</td><td class="c5">1000380</td><td class="c5">1000540</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c5">2028472</td><td class="c5">2028936</td><td class="c5">2028072</td><td class="c5">2028572</td><td class="c5">2028472</td><td class="c5">2028996</td></tr>
</table>
<a name="sec5">
<h2>5. Performance On SSD</h2></a>
<p>The same tests as in <a href="#sec2">Section 2</a> are performed again, this time using the Crucial M4 SSD with reiserfs.
This drive has been in light use for only a few weeks, so it should still be near its
original performance levels.</p>
<h3>A. Sequential Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">4,629,630 ops/sec</td>
<td class="c3"><div class="bldb" style="width:115px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">834,028 ops/sec</td>
<td class="c3"><div class="bkct" style="width:20px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">315,259 ops/sec</td>
<td class="c3"><div class="bsql" style="width:7px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">14,084,507 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">843,882 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:20px"> </div></td>
</table>
<h3>B. Random Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">143,947 ops/sec</td>
<td class="c3"><div class="bldb" style="width:69px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">99,285 ops/sec</td>
<td class="c3"><div class="bkct" style="width:48px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">83,668 ops/sec</td>
<td class="c3"><div class="bsql" style="width:40px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">723,589 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">102,669 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:49px"> </div></td>
</table>
<p>Read performance is essentially the same as for tmpfs since all of the data is
present in the filesystem cache.</p>
<h3>C. Sequential Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">477,555 ops/sec</td>
<td class="c3"><div class="bldb" style="width:338px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">335,683 ops/sec</td>
<td class="c3"><div class="bkct" style="width:237px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">51,419 ops/sec</td>
<td class="c3"><div class="bsql" style="width:36px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">494,315 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">59,637 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:42px"> </div></td>
</table>
<h3>D. Random Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">164,962 ops/sec</td>
<td class="c3"><div class="bldb" style="width:243px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">93,266 ops/sec</td>
<td class="c3"><div class="bkct" style="width:137px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">38,619 ops/sec</td>
<td class="c3"><div class="bsql" style="width:57px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">236,798 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">16,783 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:24px"> </div></td>
</table>
<p>Most of the databases perform at close to their tmpfs speeds, which is expected
since these are asynchronous writes. However, BerkeleyDB shows a large reduction in
throughput. Also, surprisingly, MDB beats LevelDB's write performance here.
Even though the writes are fully cached, the underlying storage device
still has an impact on write throughput.</p>
<h3>E. Batch Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">619,963 entries/sec</td>
<td class="c3"><div class="bldb" style="width:90px"> </div></td>
<td class="c4">(1.30x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">335,683 entries/sec</td>
<td class="c3"><div class="bkct" style="width:49px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">99,850 entries/sec</td>
<td class="c3"><div class="bsql" style="width:14px"> </div></td>
<td class="c4">(1.94x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">2,386,635 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(4.83x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">111,297 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:16px"> </div></td>
<td class="c4">(1.87x non-batched)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">228,154 entries/sec</td>
<td class="c3"><div class="bldb" style="width:279px"> </div></td>
<td class="c4">(1.38x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">93,266 entries/sec</td>
<td class="c3"><div class="bkct" style="width:114px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">53,700 entries/sec</td>
<td class="c3"><div class="bsql" style="width:65px"> </div></td>
<td class="c4">(1.39x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">286,205 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(1.21x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">16,254 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:19px"> </div></td>
<td class="c4">(0.97x non-batched)</td></tr>
</table>
<h3>F. Synchronous Writes</h3>
<p>Here the difference between SSD and tmpfs is made obvious. Note that due
to the overall slowness of these operations, they were only performed 1000 times each.</p>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">342 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.0007x asynch)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">69 ops/sec</td>
<td class="c3"><div class="bkct" style="width:69px"> </div></td>
<td class="c4">(0.0002x asynch)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">114 ops/sec</td>
<td class="c3"><div class="bsql" style="width:114px"> </div></td>
<td class="c4">(0.0022x asynch)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">149 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:149px"> </div></td>
<td class="c4">(0.0003x asynch)</td></tr>
<tr><td class="c1">MDB, no MetaSync</td>
<td class="c2">328 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:330px"> </div></td>
<td class="c4">(0.0007x asynch)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">300 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:302px"> </div></td>
<td class="c4">(0.0050x asynch)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">342 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.0021x asynch)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">67 ops/sec</td>
<td class="c3"><div class="bkct" style="width:68px"> </div></td>
<td class="c4">(0.0007x asynch)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">114 ops/sec</td>
<td class="c3"><div class="bsql" style="width:116px"> </div></td>
<td class="c4">(0.0029x asynch)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">148 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:151px"> </div></td>
<td class="c4">(0.0006x asynch)</td></tr>
<tr><td class="c1">MDB, no MetaSync</td>
<td class="c2">322 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:329px"> </div></td>
<td class="c4">(0.0013x asynch)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">291 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:297px"> </div></td>
<td class="c4">(0.0173x asynch)</td></tr>
</table>
<p>The slowness of the SSD overshadows any difference between sequential and random
write performance here.</p>
<p>MDB's synchronous writes are slower because by default it does two separate syncs per commit -
one for the data, and one for the meta page update. Generally there is a large seek penalty for
the sync of the meta page, although that should not be an issue with an SSD.
Using MDB's NOMETASYNC option will skip the explicit sync of the
meta page update. Omitting the explicit sync of the meta page means the meta page
update for one transaction may not be sync'd along with that transaction but definitely will be
sync'd by the commit of the next following transaction.
In this case, if the system crashes, the last committed transaction may be lost. Some apps
can tolerate this reduced level of reliability. Also, on filesystems which support
ordered writes, no transactions will be lost.</p>
<a name="sec6">
<h2>6. Performance Using More Memory</h2></a>
<p>We increased the overall cache size for each database to 128 MB, as in <a href="#sec3">Section 3</a>.
The "baseline" in these tests refers to the values from <a href="#sec5">Section 5</a>.</p>
<h3>A. Sequential Reads</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">4,672,897 ops/sec</td>
<td class="c3"><div class="bldb" style="width:116px"> </div></td>
<td class="c4">(1.01x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">1,246,883 ops/sec</td>
<td class="c3"><div class="bkct" style="width:30px"> </div></td>
<td class="c4">(1.50x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">438,982 ops/sec</td>
<td class="c3"><div class="bsql" style="width:10px"> </div></td>
<td class="c4">(1.39x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">14,084,507 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">819,672 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:20px"> </div></td>
<td class="c4">(0.97x baseline)</td></tr>
</table>
<h3>B. Random Reads</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">142,857 ops/sec</td>
<td class="c3"><div class="bldb" style="width:69px"> </div></td>
<td class="c4">(0.99x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">193,386 ops/sec</td>
<td class="c3"><div class="bkct" style="width:93px"> </div></td>
<td class="c4">(1.95x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">104,690 ops/sec</td>
<td class="c3"><div class="bsql" style="width:50px"> </div></td>
<td class="c4">(1.25x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">723,589 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">160,077 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:77px"> </div></td>
<td class="c4">(1.56x baseline)</td></tr>
</table>
<h3>C. Sequential Writes</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">483,793 ops/sec</td>
<td class="c3"><div class="bldb" style="width:342px"> </div></td>
<td class="c4">(0.92x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">323,625 ops/sec</td>
<td class="c3"><div class="bkct" style="width:229px"> </div></td>
<td class="c4">(0.96x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">49.925 ops/sec</td>
<td class="c3"><div class="bsql" style="width:35px"> </div></td>
<td class="c4">(0.97x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">494,315 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">71,726 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:50px"> </div></td>
<td class="c4">(1.20x baseline)</td></tr>
</table>
<h3>D. Random Writes</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">134,608 ops/sec</td>
<td class="c3"><div class="bldb" style="width:198px"> </div></td>
<td class="c4">(0.82x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">180,636 ops/sec</td>
<td class="c3"><div class="bkct" style="width:266px"> </div></td>
<td class="c4">(1.94x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">40,050 ops/sec</td>
<td class="c3"><div class="bsql" style="width:59px"> </div></td>
<td class="c4">(1.04x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">236,798 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">53,333 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:78px"> </div></td>
<td class="c4">(3.18x baseline)</td></tr>
</table>
<h3>E. Batch Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">610,128 entries/sec</td>
<td class="c3"><div class="bldb" style="width:89px"> </div></td>
<td class="c4">(1.26x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">323,625 entries/sec</td>
<td class="c3"><div class="bkct" style="width:47px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">94,171 entries/sec</td>
<td class="c3"><div class="bsql" style="width:13px"> </div></td>
<td class="c4">(1.89x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">2,386,635 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(4.83x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">108,108 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:15px"> </div></td>
<td class="c4">(1.51x non-batched)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">228,624 entries/sec</td>
<td class="c3"><div class="bldb" style="width:279px"> </div></td>
<td class="c4">(1.70x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">180,636 entries/sec</td>
<td class="c3"><div class="bkct" style="width:220px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">56,145 entries/sec</td>
<td class="c3"><div class="bsql" style="width:68px"> </div></td>
<td class="c4">(1.40x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">286,205 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(1.21x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">76,622 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:93px"> </div></td>
<td class="c4">(1.44x non-batched)</td></tr>
</table>
<h3>F. Synchronous Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">348 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.0007x asynch)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">69 ops/sec</td>
<td class="c3"><div class="bkct" style="width:69px"> </div></td>
<td class="c4">(0.0002x asynch)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">117 ops/sec</td>
<td class="c3"><div class="bsql" style="width:117px"> </div></td>
<td class="c4">(0.0023x asynch)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">149 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:149px"> </div></td>
<td class="c4">(0.0003x asynch)</td></tr>
<tr><td class="c1">MDB, no MetaSync</td>
<td class="c2">328 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:330px"> </div></td>
<td class="c4">(0.0007x asynch)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">299 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:300px"> </div></td>
<td class="c4">(0.0042x asynch)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">349 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.0026x asynch)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">67 ops/sec</td>
<td class="c3"><div class="bkct" style="width:67px"> </div></td>
<td class="c4">(0.0004x asynch)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">118 ops/sec</td>
<td class="c3"><div class="bsql" style="width:118px"> </div></td>
<td class="c4">(0.0029x asynch)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">148 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:148px"> </div></td>
<td class="c4">(0.0006x asynch)</td></tr>
<tr><td class="c1">MDB, no MetaSync</td>
<td class="c2">322 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:329px"> </div></td>
<td class="c4">(0.0013x asynch)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">297 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:297px"> </div></td>
<td class="c4">(0.0056x asynch)</td></tr>
</table>
<p>The impact of the increased cache is inconsistent at best.</p>
<a name="sec7">
<h2>7. Performance Using Large Values</h2></a>
<p>This is the same as the test in <a href="#sec4">Section 4</a>, using the SSD.</p>
<h3>A. Sequential Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">355,240 ops/sec</td>
<td class="c3"><div class="bldb" style="width:4px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">16,384 ops/sec</td>
<td class="c3"><div class="bkct" style="width:0px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">7,163 ops/sec</td>
<td class="c3"><div class="bsql" style="width:0px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">30,303,030 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">9,144 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:0px"> </div></td>
</table>
<h3>B. Random Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">19,245 ops/sec</td>
<td class="c3"><div class="bldb" style="width:3px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">14,639 ops/sec</td>
<td class="c3"><div class="bkct" style="width:3px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">6,943 ops/sec</td>
<td class="c3"><div class="bsql" style="width:1px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">1,697,793 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">8,652 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:1px"> </div></td>
</table>
<p>The read results are about the same as for tmpfs.</p>
<h3>C. Sequential Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">761 ops/sec</td>
<td class="c3"><div class="bldb" style="width:27px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">3,449 ops/sec</td>
<td class="c3"><div class="bkct" style="width:123px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">1,080 ops/sec</td>
<td class="c3"><div class="bsql" style="width:38px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">9,762 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">450 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:16px"> </div></td>
</table>
<h3>D. Random Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">126 ops/sec</td>
<td class="c3"><div class="bldb" style="width:4px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">2,578 ops/sec</td>
<td class="c3"><div class="bkct" style="width:93px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">948 ops/sec</td>
<td class="c3"><div class="bsql" style="width:34px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">9,687 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">435 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:15px"> </div></td>
</table>
<h3>E. Batch Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">1,575 entries/sec</td>
<td class="c3"><div class="bldb" style="width:55px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">3,449 entries/sec</td>
<td class="c3"><div class="bkct" style="width:120px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">1107 entries/sec</td>
<td class="c3"><div class="bsql" style="width:38px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">9,996 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">454 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:15px"> </div></td>
</table>
<h4>Random Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">1,574 entries/sec</td>
<td class="c3"><div class="bldb" style="width:55px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">2,578 entries/sec</td>
<td class="c3"><div class="bkct" style="width:90px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">970 entries/sec</td>
<td class="c3"><div class="bsql" style="width:34px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">9,922 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">441 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:15px"> </div></td>
</table>
<p>MDB dominates the asynchronous write tests.</p>
<h3>F. Synchronous Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">110 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">34 ops/sec</td>
<td class="c3"><div class="bkct" style="width:106px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">83 ops/sec</td>
<td class="c3"><div class="bsql" style="width:264px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">91 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:289px"> </div></td>
<tr><td class="c1">MDB, no MetaSync</td>
<td class="c2">93 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:294px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">100 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:319px"> </div></td>
</table>
<h4>Random Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">92 ops/sec</td>
<td class="c3"><div class="bldb" style="width:302px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">30 ops/sec</td>
<td class="c3"><div class="bkct" style="width:98px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">80 ops/sec</td>
<td class="c3"><div class="bsql" style="width:262px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">66 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:215px"> </div></td>
<tr><td class="c1">MDB, no MetaSync</td>
<td class="c2">97 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:316px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">107 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:350px"> </div></td>
</table>
<p>The benefit of MDB's NOMETASYNC option is marginal here; most of the time
required for these operations is simply in writing the 100,000 byte data values.</p>
<a name="sec8">
<h2>8. Performance On HDD</h2></a>
<p>The same tests as in <a href="#sec2">Section 2</a> are performed again,
this time using the Seagate ST9500420AS HDD with EXT2 fs. The EXT2 filesystem
was chosen based on our <a href="july/#sec11">
previous survey of multiple filesystems</a>.
</p>
<h3>A. Sequential Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">4,385,965 ops/sec</td>
<td class="c3"><div class="bldb" style="width:108px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">823,723 ops/sec</td>
<td class="c3"><div class="bkct" style="width:20px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">306,937 ops/sec</td>
<td class="c3"><div class="bsql" style="width:7px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">14,084,507 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">825,083 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:20px"> </div></td>
</table>
<h3>B. Random Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">135,925 ops/sec</td>
<td class="c3"><div class="bldb" style="width:63px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">101,041 ops/sec</td>
<td class="c3"><div class="bkct" style="width:47px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">81,579 ops/sec</td>
<td class="c3"><div class="bsql" style="width:38px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">746,826 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">102,323 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:47px"> </div></td>
</table>
<p>Read performance is essentially the same as the previous tests since all of the data is
present in the filesystem cache.</p>
<h3>C. Sequential Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">497,265 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">337,952 ops/sec</td>
<td class="c3"><div class="bkct" style="width:237px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">49,383 ops/sec</td>
<td class="c3"><div class="bsql" style="width:34px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">488,281 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:343px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">68,507 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:48px"> </div></td>
</table>
<h3>D. Random Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">172,058 ops/sec</td>
<td class="c3"><div class="bldb" style="width:258px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">90,827 ops/sec</td>
<td class="c3"><div class="bkct" style="width:136px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">19,623 ops/sec</td>
<td class="c3"><div class="bsql" style="width:29px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">233,155 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">27,625 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:41px"> </div></td>
</table>
<h3>E. Batch Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">517,063 entries/sec</td>
<td class="c3"><div class="bldb" style="width:72px"> </div></td>
<td class="c4">(1.04x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">337,952 entries/sec</td>
<td class="c3"><div class="bkct" style="width:47px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">97,267 entries/sec</td>
<td class="c3"><div class="bsql" style="width:13px"> </div></td>
<td class="c4">(1.97x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">2,487,562 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(5.09x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">118,329 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:16px"> </div></td>
<td class="c4">(1.73x non-batched)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">226,244 entries/sec</td>
<td class="c3"><div class="bldb" style="width:276px"> </div></td>
<td class="c4">(1.31x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">90,827 entries/sec</td>
<td class="c3"><div class="bkct" style="width:110px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">25,825 entries/sec</td>
<td class="c3"><div class="bsql" style="width:31px"> </div></td>
<td class="c4">(1.32x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">286,451 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(1.23x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">26,190 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:32px"> </div></td>
<td class="c4">(0.95x non-batched)</td></tr>
</table>
<h3>F. Synchronous Writes</h3>
<p>As before, only 1000 operations are performed, due to the slowness of these tests.
The HDD's cache was not disabled for these tests.</p>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">1260 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.0025x asynch)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">30 ops/sec</td>
<td class="c3"><div class="bkct" style="width:8px"> </div></td>
<td class="c4">(0.0001x asynch)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">114 ops/sec</td>
<td class="c3"><div class="bsql" style="width:31px"> </div></td>
<td class="c4">(0.0023x asynch)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">364 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:101px"> </div></td>
<td class="c4">(0.0007x asynch)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">733 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:203px"> </div></td>
<td class="c4">(0.0107x asynch)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">1291 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.0075x asynch)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">28 ops/sec</td>
<td class="c3"><div class="bkct" style="width:7px"> </div></td>
<td class="c4">(0.0003x asynch)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">112 ops/sec</td>
<td class="c3"><div class="bsql" style="width:30px"> </div></td>
<td class="c4">(0.0057x asynch)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">297 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:80px"> </div></td>
<td class="c4">(0.0013x asynch)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">704 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:190px"> </div></td>
<td class="c4">(0.0255x asynch)</td></tr>
</table>
<p>The slowness of the HDD overshadows any difference between sequential and random
write performance here. LevelDB seems to benefit more from the HDD's caching than
any of the other databases.</p>
<p>MDB's NOMETASYNC option made no discernible difference here so those results
are omitted. Ultimately the seek
overhead cannot be avoided using a single storage device. Ideally the meta pages
should be isolated in a separate file that can be stored on a separate spindle to
avoid the seek overhead.</p>
<a name="sec9">
<h2>9. Performance Using More Memory</h2></a>
<p>We increased the overall cache size for each database to 128 MB, as in <a href="#sec3">Section 3</a>.
The "baseline" in these tests refers to the values from <a href="#sec8">Section 8</a>.</p>
<h3>A. Sequential Reads</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">4,524,887 ops/sec</td>
<td class="c3"><div class="bldb" style="width:112px"> </div></td>
<td class="c4">(1.03x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">1,246,883 ops/sec</td>
<td class="c3"><div class="bkct" style="width:30px"> </div></td>
<td class="c4">(1.51x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">430,108 ops/sec</td>
<td class="c3"><div class="bsql" style="width:10px"> </div></td>
<td class="c4">(1.40x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">14,084,507 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">572,082 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:14px"> </div></td>
<td class="c4">(0.69x baseline)</td></tr>
</table>
<h3>B. Random Reads</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">137,231 ops/sec</td>
<td class="c3"><div class="bldb" style="width:64px"> </div></td>
<td class="c4">(1.01x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">192,456 ops/sec</td>
<td class="c3"><div class="bkct" style="width:90px"> </div></td>
<td class="c4">(1.90x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">104,102 ops/sec</td>
<td class="c3"><div class="bsql" style="width:48px"> </div></td>
<td class="c4">(1.28x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">746,826 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">134,626 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:63px"> </div></td>
<td class="c4">(1.32x baseline)</td></tr>
</table>
<h3>C. Sequential Writes</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">492,854 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.99x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">327,225 ops/sec</td>
<td class="c3"><div class="bkct" style="width:232px"> </div></td>
<td class="c4">(0.97x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">48,065 ops/sec</td>
<td class="c3"><div class="bsql" style="width:34px"> </div></td>
<td class="c4">(0.97x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">488,281 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:346px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">58,258 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:41px"> </div></td>
<td class="c4">(0.85x baseline)</td></tr>
</table>
<h3>D. Random Writes</h3>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">176,929 ops/sec</td>
<td class="c3"><div class="bldb" style="width:265px"> </div></td>
<td class="c4">(1.03 baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">181,127 ops/sec</td>
<td class="c3"><div class="bkct" style="width:271px"> </div></td>
<td class="c4">(1.99x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">19,085 ops/sec</td>
<td class="c3"><div class="bsql" style="width:28px"> </div></td>
<td class="c4">(0.97x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">233,155 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">46,990 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:70px"> </div></td>
<td class="c4">(1.70x baseline)</td></tr>
</table>
<h3>E. Batch Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">553,097 entries/sec</td>
<td class="c3"><div class="bldb" style="width:77px"> </div></td>
<td class="c4">(1.12x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">327,225 entries/sec</td>
<td class="c3"><div class="bkct" style="width:46px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">94,697 entries/sec</td>
<td class="c3"><div class="bsql" style="width:13px"> </div></td>
<td class="c4">(1.97x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">2,487,562 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(5.09x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">93,266 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:13px"> </div></td>
<td class="c4">(1.60x non-batched)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">229,095 entries/sec</td>
<td class="c3"><div class="bldb" style="width:279px"> </div></td>
<td class="c4">(1.29x non-batched)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">181,127 entries/sec</td>
<td class="c3"><div class="bkct" style="width:221px"> </div></td>
<td class="c4">(non-batched)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">28,129 entries/sec</td>
<td class="c3"><div class="bsql" style="width:34px"> </div></td>
<td class="c4">(1.47x non-batched)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">286,451 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<td class="c4">(1.23x non-batched)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">64,708 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:79px"> </div></td>
<td class="c4">(1.38x non-batched)</td></tr>
</table>
<h3>F. Synchronous Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">1298 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.0026x asynch)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">29 ops/sec</td>
<td class="c3"><div class="bkct" style="width:7px"> </div></td>
<td class="c4">(0.0001x asynch)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">115 ops/sec</td>
<td class="c3"><div class="bsql" style="width:30px"> </div></td>
<td class="c4">(0.0024x asynch)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">364 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:98px"> </div></td>
<td class="c4">(0.0007x asynch)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">733 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:197px"> </div></td>
<td class="c4">(0.0126x asynch)</td></tr>
</table>
<h4>Random Writes</h4>
<table class="bn">
<tr><td class="c1">LevelDB</td>
<td class="c2">1323 ops/sec</td>
<td class="c3"><div class="bldb" style="width:350px"> </div></td>
<td class="c4">(0.0075x baseline)</td></tr>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">29 ops/sec</td>
<td class="c3"><div class="bkct" style="width:7px"> </div></td>
<td class="c4">(0.0002x baseline)</td></tr>
<tr><td class="c1">SQLite3</td>
<td class="c2">113 ops/sec</td>
<td class="c3"><div class="bsql" style="width:29px"> </div></td>
<td class="c4">(0.0059x baseline)</td></tr>
<tr><td class="c1">MDB</td>
<td class="c2">297 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:78px"> </div></td>
<td class="c4">(0.0013x baseline)</td></tr>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">649 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:171px"> </div></td>
<td class="c4">(0.0138x baseline)</td></tr>
</table>
<a name="sec10">
<h2>10. Performance Using Large Values</h2></a>
<p>This is the same as the test in <a href="#sec4">Section 4</a>, using the HDD.</p>
<h3>A. Sequential Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">373,413 ops/sec</td>
<td class="c3"><div class="bldb" style="width:4px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">15,910 ops/sec</td>
<td class="c3"><div class="bkct" style="width:0px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">7,634 ops/sec</td>
<td class="c3"><div class="bsql" style="width:0px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">28,571,429 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">9,443 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:0px"> </div></td>
</table>
<h3>B. Random Reads</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">20,831 ops/sec</td>
<td class="c3"><div class="bldb" style="width:4px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">14,167 ops/sec</td>
<td class="c3"><div class="bkct" style="width:2px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">7,210 ops/sec</td>
<td class="c3"><div class="bsql" style="width:1px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">1,700,680 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">8,786 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:1px"> </div></td>
</table>
<p>Again, the read results are about the same as for tmpfs.</p>
<h3>C. Sequential Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">651 ops/sec</td>
<td class="c3"><div class="bldb" style="width:19px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,124 ops/sec</td>
<td class="c3"><div class="bkct" style="width:151px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">1,525 ops/sec</td>
<td class="c3"><div class="bsql" style="width:44px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">11,865 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">470 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:13px"> </div></td>
</table>
<h3>D. Random Writes</h3>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">99 ops/sec</td>
<td class="c3"><div class="bldb" style="width:2px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,000 ops/sec</td>
<td class="c3"><div class="bkct" style="width:149px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">553 ops/sec</td>
<td class="c3"><div class="bsql" style="width:16px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">11,716 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">519 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:15px"> </div></td>
</table>
<h3>E. Batch Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">759 entries/sec</td>
<td class="c3"><div class="bldb" style="width:21px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,124 entries/sec</td>
<td class="c3"><div class="bkct" style="width:147px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">1,828 entries/sec</td>
<td class="c3"><div class="bsql" style="width:52px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">12,135 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">482 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:13px"> </div></td>
</table>
<h4>Random Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">754 entries/sec</td>
<td class="c3"><div class="bldb" style="width:21px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">5,000 entries/sec</td>
<td class="c3"><div class="bkct" style="width:145px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">1,820 entries/sec</td>
<td class="c3"><div class="bsql" style="width:52px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">12,037 entries/sec</td>
<td class="c3"><div class="bmdb" style="width:350px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">474 entries/sec</td>
<td class="c3"><div class="bbdb" style="width:13px"> </div></td>
</table>
<h3>F. Synchronous Writes</h3>
<h4>Sequential Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">119 ops/sec</td>
<td class="c3"><div class="bldb" style="width:305px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">21 ops/sec</td>
<td class="c3"><div class="bkct" style="width:53px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">107 ops/sec</td>
<td class="c3"><div class="bsql" style="width:275px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">119 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:307px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">136 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:350px"> </div></td>
</table>
<h4>Random Writes</h4>
<table class="bn bnbase">
<tr><td class="c1">LevelDB</td>
<td class="c2">117 ops/sec</td>
<td class="c3"><div class="bldb" style="width:300px"> </div></td>
<tr><td class="c1">Kyoto TreeDB</td>
<td class="c2">18 ops/sec</td>
<td class="c3"><div class="bkct" style="width:47px"> </div></td>
<tr><td class="c1">SQLite3</td>
<td class="c2">107 ops/sec</td>
<td class="c3"><div class="bsql" style="width:275px"> </div></td>
<tr><td class="c1">MDB</td>
<td class="c2">113 ops/sec</td>
<td class="c3"><div class="bmdb" style="width:290px"> </div></td>
<tr><td class="c1">BerkeleyDB</td>
<td class="c2">136 ops/sec</td>
<td class="c3"><div class="bbdb" style="width:350px"> </div></td>
</table>
<p>The slowness of the HDD makes most of the database implementations perform about
the same. As before, Kyoto Cabinet is much slower than the rest.</p>
<p>The raw data for all of these tests is also available.
<a href="out3.ram">tmpfs</a>, <a href="out3.ssd">SSD</a>, and <a href="out3.hdd">HDD</a>.
The results are also tabulated in an OpenOffice spreadsheet for further analysis
<a href="MDB-bench-data3.ods">here</a>.
The raw data includes additional tests (e.g. reverse sequential read) which were
omitted from this already lengthy report for space reasons.
</p>
</body>
</html>
<!--
FILE ARCHIVED ON 01:56:34 Mar 04, 2016 AND RETRIEVED FROM THE
INTERNET ARCHIVE ON 00:34:22 Sep 06, 2023.
JAVASCRIPT APPENDED BY WAYBACK MACHINE, COPYRIGHT INTERNET ARCHIVE.
ALL OTHER CONTENT MAY ALSO BE PROTECTED BY COPYRIGHT (17 U.S.C.
SECTION 108(a)(3)).
-->
<!--
playback timings (ms):
captures_list: 83.552 (11)
exclusion.robots: 0.196
exclusion.robots.policy: 0.186
cdx.remote: 0.063
esindex: 0.009
LoadShardBlock: 40.273 (3)
PetaboxLoader3.datanode: 86.039 (4)
load_resource: 109.531
PetaboxLoader3.resolve: 55.262
-->